Coding with Constraints: Minimum Distance Bounds and Systematic Constructions

We examine an error-correcting coding framework in which each coded symbol is constrained to be a function of a fixed subset of the message symbols. With an eye toward distributed storage applications, we seek to design systematic codes with good minimum distance that can be decoded efficiently. On this note, we provide theoretical bounds on the minimum distance of such a code based on the coded symbol constraints. We refine these bounds in the case where we demand a systematic linear code. Finally, we provide conditions under which each of these bounds can be achieved by choosing our code to be a subcode of a Reed-Solomon code, allowing for efficient decoding. This problem has been considered in multisource multicast network error correction. The problem setup is also reminiscent of locally repairable codes.Comment: Submitted to ISIT 201

Error-Correcting Codes for Networks, Storage and Computation

The advent of the information age has bestowed upon us three challenges related to the way we deal with data. Firstly, there is an unprecedented demand for transmitting data at high rates. Secondly, the massive amounts of data being collected from various sources needs to be stored across time. Thirdly, there is a need to process the data collected and perform computations on it in order to extract meaningful information out of it. The interconnected nature of modern systems designed to perform these tasks has unraveled new difficulties when it comes to ensuring their resilience against sources of performance degradation. In the context of network communication and distributed data storage, system-level noise and adversarial errors have to be combated with efficient error correction schemes. In the case of distributed computation, the heterogeneous nature of computing clusters can potentially diminish the speedups promised by parallel algorithms, calling for schemes that mitigate the effect of slow machines and communication delay. This thesis addresses the problem of designing efficient fault tolerance schemes for the three scenarios just described. In the network communication setting, a family of multiple-source multicast networks that employ linear network coding is considered for which capacity-achieving distributed error-correcting codes, based on classical algebraic constructions, are designed. The codes require no coordination between the source nodes and are end to end: except for the source nodes and the destination node, the operation of the network remains unchanged. In the context of data storage, balanced error-correcting codes are constructed so that the encoding effort required is balanced out across the storage nodes. In particular, it is shown that for a fixed row weight, any cyclic Reed-Solomon code possesses a generator matrix in which the number of nonzeros is the same across the columns. In the balanced and sparsest case, where each row of the generator matrix is a minimum distance codeword, the maximal encoding time over the storage nodes is minimized, a property that is appealing in write-intensive settings. Analogous constructions are presented for a locally recoverable code construction due to Tamo and Barg. Lastly, the problem of mitigating stragglers in a distributed computation setup is addressed, where a function of some dataset is computed in parallel. Using Reed-Solomon coding techniques, a scheme is proposed that allows for the recovery of the function under consideration from the minimum number of machines possible. The only assumption made on the function is that it is additively separable, which renders the scheme useful in distributed gradient descent implementations. Furthermore, a theoretical model for the run time of the scheme is presented. When the return time of the machines is modeled probabilistically, the model can be used to optimally pick the scheme's parameters so that the expected computation time is minimized. The recovery is performed using an algorithm that runs in quadratic time and linear space, a notable improvement compared to state-of-the-art schemes. The unifying theme of the three scenarios is the construction of error-correcting codes whose encoding functions adhere to certain constraints. It is shown that in many cases, these constraints can be satisfied by classical constructions. As a result, the schemes presented are deterministic, operate over small finite fields and can be decoded using efficient algorithms.</p

Distributed Reed-Solomon Codes for Simple Multiple Access Networks

We consider a simple multiple access network in which a destination node receives information from multiple sources via a set of relay nodes. Each relay node has access to a subset of the sources, and is connected to the destination by a unit capacity link. We also assume that $z$ of the relay nodes are adversarial. We propose a computationally efficient distributed coding scheme and show that it achieves the full capacity region for up to three sources. Specifically, the relay nodes encode in a distributed fashion such that the overall codewords received at the destination are codewords from a single Reed-Solomon code.Comment: 12 pages, 1 figur

Improving Distributed Gradient Descent Using Reed-Solomon Codes

Today's massively-sized datasets have made it necessary to often perform computations on them in a distributed manner. In principle, a computational task is divided into subtasks which are distributed over a cluster operated by a taskmaster. One issue faced in practice is the delay incurred due to the presence of slow machines, known as \emph{stragglers}. Several schemes, including those based on replication, have been proposed in the literature to mitigate the effects of stragglers and more recently, those inspired by coding theory have begun to gain traction. In this work, we consider a distributed gradient descent setting suitable for a wide class of machine learning problems. We adapt the framework of Tandon et al. (arXiv:1612.03301) and present a deterministic scheme that, for a prescribed per-machine computational effort, recovers the gradient from the least number of machines $f$ theoretically permissible, via an $O(f^2)$ decoding algorithm. We also provide a theoretical delay model which can be used to minimize the expected waiting time per computation by optimally choosing the parameters of the scheme. Finally, we supplement our theoretical findings with numerical results that demonstrate the efficacy of the method and its advantages over competing schemes

Representations of the Multicast Network Problem

We approach the problem of linear network coding for multicast networks from different perspectives. We introduce the notion of the coding points of a network, which are edges of the network where messages combine and coding occurs. We give an integer linear program that leads to choices of paths through the network that minimize the number of coding points. We introduce the code graph of a network, a simplified directed graph that maintains the information essential to understanding the coding properties of the network. One of the main problems in network coding is to understand when the capacity of a multicast network is achieved with linear network coding over a finite field of size q. We explain how this problem can be interpreted in terms of rational points on certain algebraic varieties.Comment: 24 pages, 19 figure

Balanced Reed-Solomon codes

We consider the problem of constructing linear MDS error-correcting codes with generator matrices that are sparsest and balanced. In this context, sparsest means that every row has the least possible number of non-zero entries, and balanced means that every column contains the same number of non-zero entries. Codes with this structure minimize the maximal computation time of computing any code symbol, a property that is appealing to systems where computational load-balancing is critical. The problem was studied before by Dau et al. where it was shown that there always exists an MDS code over a sufficiently large field such that its generator matrix is both sparsest and balanced. However, the construction is not explicit and more importantly, the resulting MDS codes do not lend themselves to efficient error correction. With an eye towards explicit constructions with efficient decoding, we show in this paper that the generator matrix of a cyclic Reed-Solomon code of length n and dimension k can always be transformed to one that is both sparsest and balanced, for all parameters n and k where k/n (n − k + 1) is an integer

(Almost) practical tree codes

We consider the problem of stabilizing an unstable plant driven by bounded noise over a digital noisy communication link, a scenario at the heart of networked control. To stabilize such a plant, one needs real-time encoding and decoding with an error probability profile that decays exponentially with the decoding delay. The works of Schulman and Sahai over the past two decades have developed the notions of tree codes and anytime capacity, and provided the theoretical framework for studying such problems. Nonetheless, there has been little practical progress in this area due to the absence of explicit constructions of tree codes with efficient encoding and decoding algorithms. Recently, linear time-invariant tree codes were proposed to achieve the desired result under maximum-likelihood decoding. In this work, we take one more step towards practicality, by showing that these codes can be efficiently decoded using sequential decoding algorithms, up to some loss in performance (and with some practical complexity caveats). We supplement our theoretical results with numerical simulations that demonstrate the effectiveness of the decoder in a control system setting

Distributed gabidulin codes for multiple-source network error correction

We consider the problem of noncoherent error correction in multiple-source multicast networks. We show that a distributed network error-correcting code can be designed as a subcode of a Gabidulin code. This allows the destination nodes to utilize any decoder designed using rank-metric. We show that this achieves the full capacity region of any network with up to three messages, with optimal scaling in field size

Das Solarhaus-Experiment der Fachhochschule Luebeck Entwicklungen, Ergebnisse, Chancen

SIGLETIB Hannover: FR 3054 / FIZ - Fachinformationszzentrum Karlsruhe / TIB - Technische InformationsbibliothekDEGerman

Balanced and sparse Tamo-Barg codes

We construct balanced and sparse generator matrices for Tamo and Barg's Locally Recoverable Codes (LRCs). More specifically, for a cyclic Tamo-Barg code of length n, dimension k and locality r, we show how to deterministically construct a generator matrix where the number of nonzeros in any two columns differs by at most one, and where the weight of every row is d + r − 1, where d is the minimum distance of the code. Since LRCs are designed mainly for distributed storage systems, the results presented in this work provide a computationally balanced and efficient encoding scheme for these codes. The balanced property ensures that the computational effort exerted by any storage node is essentially the same, whilst the sparse property ensures that this effort is minimal. The work presented in this paper extends a similar result previously established for Reed-Solomon (RS) codes, where it is now known that any cyclic RS code possesses a generator matrix that is balanced as described, but is sparsest, meaning that each row has d nonzeros